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[57] ABSTRACT 

An apparatus and method for post processing image data 
which previously was encoded using a discrete cosine 
transform in order to remove resulting blocking effects. 
Possible blocking points are examined to determine whether 
blocking points exist. If blocking points are determined, an 
adjustment value for each blocking point is computed and 
proportionately utilized to adjust the adjacent and neighbor- 
ing pixels. 
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METHOD AND APPARATUS FOR blocking has beeo detected. The resulting image exhibits 

BLOCKING EFFECT REDUCTION IN minimal blocking effects while maintaining a consistent 

IMAGES sharp image. 

BACKGROUND OF THE INVENTION 5 BRIEF DESCRIPTION OF THE DRAWINGS 

1. Fieldof the Invention . ^ .9*^ fcaturcs m6 «^nU^rflhcp««miiiwn- 

tion will be apparent to one skilled m the art in light of the 

Hie present invention relates to the removal of blocking following detailed description in which: 

effects in images previously compressed and decompressed. ™~ - . , _, r 4 4 . . 

w . , , r tl _ J /. t . * . . FIG. 1 is a simplified block diagram of a system that 

More particularly, the present invention relates to the * n 4 . . r . it _ 4 . . c 

i fu i '• a - • j * 10 operates m accordance with the teachings of toe present 

removal of blocking effects in images compressed in accor- • t * 

dance with MPEG, JPEG and other DCT based image MVCOtlOD - 

coding formats. FIG. 2 is a simplified flow diagram depicting the process 

„ A ^ ' for minimizing blocking effects. 

2. Art Background ° 

m , ,. . , . , . , . FIG. 3 is a diagram illustrating blocks and block bound- 

The need to compress digital image data, whether static or is arfes ^ a jm™. 

dynamic (i.e., video images) images has dramatically * • a ■>• •« * «. j- 

• j Vk * *u j i •* c w FIG. 4 is a flow diagram illustrating one embodiment 

increased. One reason is the dramatic popularity of multi- t . , , , , . f _ , . . 

j • . . j r*. a *u • process to be performed on a block by block basis, 

media computer systems and software. Another reason is r r J 

switch to digital broadcasts of station programming. One F 10 * ^ * a simplified block diagram of a circuit that 

example of the latter is direct satellite broadcasts, such as 20 Performs a process for minimizing blocking effects, and 

DSS™. In order to minimize the bandwidth required to mG - Sb fa a simplified block diagram of an illustrative 

transmit broadcasts of programs or to minimize the amount processor system which performs the steps to reduce block- 

of storage space required to store certain images, compres- ^ effects m accordance with the teachings of the present 

sion techniques are utilized. Thus, the image data is trans- invention. 

mitted or stored in a compressed format and prior to the 25 FIG. 6a and FIG. 6b illustrate the process for a single row 

display of the image, the image data is decompressed. accross a single boundary and the result achieved. 

Examples of widely utilized compression algorithms are DETAILED DESCRIPTION 

those that comply with the MPEG (Moving Pictures Experts m . , .,.«.. 

Group) and JPEG (Joint Picture Experts Group) standards. ^ P reseDt "vention prorate a simple but effective 

. . • * . . , n apparatus and method for minimizing blocking effects that 

i^iSS^ o^ r m discrete transformed images. In the following 

ply with MPEGor JPEG standards, utmze^ansform coding ^ fof ^ of ^ numerous detai £ 

In a transform coding process an image is divided into small afe M ^ m ^ {Q ^ a ^ h of 

blocks. The transform of each block is taken the coefficients me t j5 owever , it will be apparent to one 

of which are quantized in accordance with a determined ^ ^ ^ m ^ ^ ^ details ™ M ^ 

Quantization factor q. The most popular transform is the ^ ^ ^ ^ me pr&sent In other instance5f 

discrete cosine transform (DCT). weU known electrical structures and circuits are shown in 

However, one negative effect of the process is referred to block d i agram f orm m or d cr not to obscure the present 

as the "blocking effect". By dividing the image into blocks invention unnecessarily. 

prior to encoding, discontinuities (referred to as blocking A simplified block diagram of an exemplary system that 

effects) between adjacent blocks occurs. This is represented operates in accordance with the teachings of the present 

in a displayed decompressed image by clear jumps between invention ^ illustrated in FIG. 1. To minimize on transmis- 

colors or greyscales as opposed to a smooth change. sioQ bandwidth md/oT to ^ on the amount of space 

Pre-processing and post-processing techniques are uti- required to store an image, the image is frequently formatted 

lized to minimize blocking effects. Pre-processing tech- 4S in a compressed form which utilizes discrete transforms, 

niques dictate that the originator of the image data must Popular examples of formats include those that comply with 

perform certain steps to minimize blocking effects. Post- the JPEG Joint Picture Experts Group) and MPEG (Motion 

processing techniques, although logistically better as the Picture Experts Group) standards. The system illustrated 

correction is performed after decompression, has its prob- includes the receiver/display system 10 and display 40. One 

lems. For example, one of the simplest techniques is to 50 example of a receiver/display system 10 is a Video CD 

process the decompressed image data through a low pass player. Another example is a direct satellite receiver such as 

filter. Although the blocking effects are decreased, the sharp- one manufactured by Sony Corporation. Other types of 

ness of the displayed image is negatively affected. receivers/players/storage/display systems are also contem- 
plated. 

55 The receiver 10 receives the compressed image. The 

In the method and circuit of the present invention post receiver may be one of many types of receiving devices 

processing is performed on decompressed image data to configured to receive image data. Alternately, the receiver 

detect and minimize blocking effects without affecting the may be one to receive television broadcast signals or a 

sharpness of the image. device coupled to directly to a storage unit (e.g., memory, 

First pixels subject to blocking errors are detected. Pref- 60 VCR, CD ROM or the like) to receive images retrieved from 

jerably the difference in pixel values (e.g., luminance) storage. For purposes of explanation, the image data is 

'between neighboring pixels is examined' to determine received in a format compatible with the MPEG specifica- 

whether the difference value is within a range identified to tion; however, the other types of compressed formats that 

be a blocking errors. Once a blocking error is detected, use discrete transforms to compress the data can also be 

adjustment terms are added to the pixels detected and 65 used. 

neighboring pixels in the image. The amount of adjustment Once the image data is received, the data is decoded on a 

is proportional to the number of pixels away from where block by block basis (i.e., decompressed) by decoder 20 and 
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processed by post-processor 30 in order to remove blocking 
effects created by the encoding and decoding processes. The 
modified image data subsequently displayed by display 
subsystem 40 shows an image in which the blocking effects 
are minimized. 

The process performed by post-processor 30 is generally 
described with reference to FIG. 2. In the present invention, 
the adjacent pixels at each boundary point around the block 
are examined to determine if a blocking point exists between 
the pixels, step 200. If a blocking point is identified, an 
adjustment value is determined and applied to the pixels, 
step 220. This adjustment value is applied to the adjacent 
and neighboring pixels in a graduated or manner propor- 
tional to the distance from the blocking point. By adjusting 
not only the adjacent pixels, but also the neighboring pixels, 
the adjustment result in a smoother image in the areas 
surrounding block boundaries. 

The detection process can be applied to each pair of 
adjacent pixels in the image, for example on a column by 
column or row by row basis. Preferably, to minimize pro- 
cessing time, while maintaining effectiveness, the pixels 
adjacent to block boundaries are examined. This is best 
illustrated with reference to FIG. 3. As noted earlier, the 
image 300 is divided into blocks (e.g., blocks 305, 310, 320, 
330) and the DCT and quantization process are applied to 
each block of pixels. The blocking effects occur at the 
blocking boundaries (e.g., block boundaries 340, 345, 350, 
355 of block 330). For example, if a vertical boundary 345 
is examined, the pixels in the rows affected, the adjacent 
pixels on both sides of the blocking boundary in rows 
351-358 are examined to determine if a blocking point 
exists and the adjustments are performed on the adjacent and 
predetermined neighboring pixels in the corresponding row. 
Similarly, if a horizontal boundary 340 is examined, the 
adjacent pixels in the columns 361-368 are examined to 
determine the existence of blocking points and the adjacent 
and neighboring pixels in those columns are adjusted. 

One embodiment of the process for detecting and mini- 
mizing blocking effects is described with reference to FIG. 
4. As noted earlier, the difference between adjacent pixels 
(blocking step) at each block boundary is compared to a 
pre-specified range of values indicative of a boundary point. 

The upper and lower limits of the range of values indica- 
tive of a blocking point can be determined a variety of ways. 
Preferably the upper and tower limits are selected such that 
normal minor image variations between pixels are not 
detected as blocking points and significant image variations, 
for example indicative of different objects in the image 
itself, are not detected as blocking points. The upper and 
lower limits can be pre-specified values empirically deter- 
mined. Other ways for determining the upper and lower 
limits are also contemplated. For example, the values can be 
dynamically determined. However, it is preferred that the 
lower limit value be the local variance for the neighborhood 
of pixels. 

At step 400, the local variance of the neighborhood 
around the block is determined. Preferably, the local vari- 
ance is determined for each potential location of a blocking 
point. The size of the neighborhood (i.e., the number of 
pixels) and the location of the pixels examined can vary. In 
the present example, if a vertical boundary is examined, the 
portion of a row utilized to determine a particular potential 
vertical boundary point contains the following pixels: 

x+4 x+5 x+6 x+7 j x+8 x+9 x+10 x+11 

(where | represents the location of the boundary point) 



'4,196 
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The local variance of a neighborhood of pixels can be 
defined a variety of ways. In the present embodiment the 
local variance is determined as the average of the summation 
of the absolute values of the difference between luminance 
5 values of adjacent pixels of neighborhood used. In the 
present example, the neighborhood is composed of 6 pixels 
(i+5, i+6, i+7, i+8, i+9, i+10); therefore the following 
equation is used: 

local variance=cte fcO+61 hPl i+5 I T>**bs (rfi>7fl ]-rf/+6]I 
10 }+abs (p[i+9l ]-p[/+Bl D-whsW+lOI Htf+^I DM 

where abs represents an absolute value function and p[ I ] 
represents pixel data. 

The upper limit value of can similarly be determined a 

15 variety of ways. However, in the present embodiment a 
predetermined, empirically determined value is used. This 
value is chosen to reflect a possible error that is larger than 
an error that can be caused by the quantizing of the block 
during the coding process. For example in the present 

20 embodiment a value of 32 is used. 

Once the local variance is determined and therefore the 
range of values indicative of a blocking point are identified, 
the blocking step between pixels adjacent to the block 
boundary is determined, step 410. Continuing with the 

25 above example, the difference between pixels x+7 and x+8 
is determined. Preferably, the difference value is composed 
of the difference between the luminance components in each 
pixel, although other components of the image data can be 
used. 

30 At step 420, the blocking step is compared to the range of 
values to determine whether a blocking point exists. If a 
blocking point exists, the adjustment value is determined, 
step 430. 

The adjustment value can be determined a variety of 
35 ways. The value is chosen to minimize or eliminate the 
detected blocking point. Preferably, the adjustment value is 
determined such that the step on the blocking point (i.e., at 
the blocking point) will be adjusted to the local variance in 
the corresponding region or neighborhood. Continuing with 
40 the present example, if the blocking step is greater than zero, 
the adjustment value is equal to the difference of the 
blocking step and the local variance (adjustment_value= 
blocking_step-local variance); otherwise the adjustment 
value is equal to the sum of the blocking step and local 
45 variance (adjustment_value=blocking_step+local_ 
variance). 

Once the adjustment value is determined it is added to the 
appropriate pixels to minimize or eliminate the blocking 
effect. Preferably proportionate amounts of the adjustment 
50 value is added to the pixels adjacent to the detected blocking 
point and neighboring pixels. The number of neighboring 
pixels and the proportionate amounts added can vary. 
However, it is preferred that the following pixels and pro- 
portionate amounts of the adjustment value be used to 
55 minimize the blocking effect: 

p'[i+8l ]=p[i+8][ ]-adjustment_value/2 

p'[i+7l ]=p[i+7][ ]+adjustment_value/2 

p ! [i+9J ]=p[i+9]£ ]-adjustment_value/4 

p'[i+6J ]«p[i+6][ ]+adjustment__value/4 
60 p f [i+10l ]-p[i+10l ]-adjustrnent__value/8 

pfi+51 ]«p[i+5][ ]+adjustment_value/8 

p'[i+ll][ ^pp+lll ]-adjustment_value/16 

p r [i+4l l^ti-^K ]+adjustment_value/16 
As can be seen by the above, the present example adjusts the 
65 two adjacent pixels and six neighboring pixels. The fraction 
of the adjustment value added is proportionate to the dis- 
tance from the blocking point. To simplify the circuitry used 
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to implement this functionality, fractional amounts cone- identifying a blocking point if the difference is within 

sponding to powers of 2 are used. However it is readily the p re-specified range of values; and 

apparent that other fractional amounts and percentages can for each horizontal boundary point examined; 
be used to gain the desired effect. m generating the difference between pixel data between 

A circuit inylementation is illustrated by the block dia- 5 ^ vert ically adjacent to the horizontal boundary 

gram of FIG. 5a. FIG. 5a represents one implementation; it e . 

is readily apparent that the present invention can be imple- & ^ Dg lhe m ^ nCG t0 me pre -specined range of 

mented m a dedicated microcontroller circuit, in logic or- i d 

cuitry or as software operable on a general purpose com- va ues ' m 

puter. Referring to FIG. Sa 9 the blocking image is input to 1Q identifying a blocking point if the difference is within the 
local variance block 505 and calculation of blocking step pre-specined range of values. 

block 510. The local variance block determines the local 3 - ^ method as set forth in claim 1, wherein the 

variance for use as a lower threshold as described above. pre-specified range of values indicative of a blocking point 

Block 510 calculates the blocking step using the blocking comprises an upper and lower limit, said upper and lower 

image as input. Block 515 determines, using the blocking limits selected such that normal minor image variation 

step, local variance and upper bound as inputs, whether a between pixels are not detected as blocking points and 

blocking point occurs. If a blocking point occurs, blocks significant image variations are not detected as blocking 

520, 525 calculate the adjustment, using the blocking image points. 

and identification of me blocking point as inputs and adjusts 4 - ^ method as set forth in claim 1, wherein the 

the neighboring pixels accordingly to generate the adjusted pre-specified range of values indicative of a blocking point 

image through multiplexing block 530. If a blocking point comprises an upper limit and lower limit, said lower limit 

does not occur, the original image (blocking image) is comprising the local variance of neighborhood pixels to the 

output. blocking point. 

A simplified block diagram of a processor system upon 5 * Tb e method as set forth in claim 4, wherein the local 

which this can be implemented is illustrated in FIG. 5b. The variance of neighborhood pixels is determined by averaging 

blocking image is received through input port circuitry 540 a summation of absolute values of a neighborhood of pixels 

which forwards the blocking image to processor 545. Pro- about me blocking point. 

cessor 545 includes a central processing unit 550 which <>• The method as set forth in claim 1, wherein the 

executes instructions from the memory 555 to perform the pre-specified range of values indicative of a blocking point 
steps described above to reduce blocking effects. 3Q comprises an upper limit and lower limit, said upper limit 

The advantages of the present invention are readily seen comprising an empirically determined value chosen to 

by the following example set forth in FIGS. 6a and 6b which reflect an error larger than an error that can be caused by 

illustrate the process for a single row across a block bound- quantization of the block during the quantization process, 
ary. The process is preferably performed for each row and 7 * The method as set forth in claim 1, wherein the step of 

column affected by the block boundaries. determining an adjustment value comprises the step of 

The invention has been described in conjunction with the generating the adjustment value such that the difference 

preferred embodiment. It is evident that numerous between pixel data of pixels adjacent to the blocking point 

alternatives, modifications, variations and uses will be ^ adjusted to a local variance of a neighborhood of pixels 

apparent to those skilled in the art in light of the foregoing about mc blocking point. 

description. 8- Th c method as set forth in claim 1, wherein the step of 

What is claimed is: 40 determining the adjustment value comprises the steps of: 

1. A method for minimizing blocking effects in a blocking if a first difference between pixel values of pixels adj acent 
image caused by prior performance of a discrete cosine to the blocking point is greater than zero, computing the 
transform (DCT) and quantization process, said blocking adjustment value to be a second difference between the 
image described by blocking image data comprising blocks 4S first difference and a local variance, wherein the local 
of pixel data, said method comprising the steps of: variance of neighborhood pixels is determined by aver- 

determining whether at least one blocking point occurs aging a summation of absolute values of a neighbor- 

respectively at boundary points of a block based upon hood of pixels about the blocking point; and 

a pre-specified range of values indicative of a blocking if the first difference is less than or equal to zero, com- 
point; 50 puling the adjustment value to be a sum of the first 

if at least one blocking point is identified; and difference and the local variance, 

for each blocking point identified; 9. The method as set forth in claim 1, wherein the step of 

determining an adjustment value based upon a blocking adjusting comprises adjusting adjacent and neighboring pix- 

step and local variance; and els bv proportionate amounts of the adjustment value, 
adjusting blocking data corresponding to pixel loca- 55 10 ^ method as set forth in claim 1, wherein the step 

tions adjacent to the blocking point and neighboring of adjusting comprises the step of adjusting adjacent and 

pixels to the adjacent pixels. neighboring pixels by proportionate amounts of the adjust- 

2. The method as set forth in claim 1, wherein boundary ment value proportionate to a distance each pixel of the 
points comprise vertical boundary points and horizontal adjacent and neighboring pixel is from the blocking point, 
boundary points and the step of determining whether at least 60 11- An apparatus for minimizing blocking effects in a 
one blocking point occurs comprises the steps of: blocking image caused by prior performance of a discrete 

for each vertical boundary point examined; transform (DCT) and quantization process, said 

generating the difference between pixel data between blocking image described by blocking image data compns- 

pixels horizontally adjacent to the vertical boundary m § blocks of P» el data » a PP a r a *us comprising: 
point, 6 5 an input device for receiving blocking image data; 
comparing the difference to the pre-specified range of a processor coupled to the input device, said processor 
values indicative of a blocking point, and comprising a central processing unit and memory, said 
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processor determining whether at least one blocking comprising the local variance of neighborhood pixels to the 

point occurs respectively at boundary points of a block blocking point. 

based upon a pre-specified range of values indicative of 16. The apparatus as set forth in claim 15, wherein said 

a blocking point; and if at least one blocking point is processor determines the local variance of neighborhood 

determined, for each blocking point identified, deter- 5 pixels by averaging a summation of absolute values of a 

mining an adjustment value based upon a blocking step neighborhood of pixels about the blocking point, 

and local variance and adjusting pixel data correspond- 17. The apparatus as set forth in claim 11, wherein the 

ing to pixel locations adjacent to the blocking point and pre-specified range of values indicative of a blocking point 

neighboring pixels to the adjacent pixels. comprises an upper limit and lower limit, said upper limit 

12. The apparatus as set forth in claim 11, further com- 10 comprising an empirically determined value chosen to 
prising a display device coupled to the processor to display reflect an error larger than an error that can be caused by 
pixel data output by the processor. quantization of the block during the quantization process. 

13. The apparatus as set forth in claim 11, wherein 18. The apparatus as set forth in claim 11, wherein the 
boundary points comprise vertical boundary points and processor determines the adjustment value such that the 
horizontal boundary points and the processor determines 15 difference between pixel data of pixels adjacent to the 
whether at least one blocking point occurs by examining blocking point is adjusted to a local variance of a neighbor- 
each vertical boundary point by generating the difference hood of pixels about the blocking point. 

between pixel data between pixels horizontally adjacent to 19. The apparatus as set forth in claim 11, wherein the 

the vertical boundary point, comparing the difference to the processor determines the adjustment value to be a second 

pre-specified range of values indicative of a blocking point, 20 difference between the first difference and a local variance if 

and identifying a blocking point if the difference is within a first difference between pixel values of pixels adjacent to 

the pre-specified range of values; and examining each hori- the blocking point is greater than zero, and determines the 

zontal boundary point and generating the difference between adjustment value to be a sum of the first difference and the 

pixel data between pixels vertically adjacent to the horizon- local variance if the first difference is less than or equal to 

tal boundary point, comparing the difference to the pre- 25 zero, wherein the local variance of neighborhood pixels is 

specified range of values, and identifying a blocking point if determined by averaging a summation of absolute values of 

the difference is within the pre-specified range of values. a neighborhood of pixels about the blocking point. 

14. The apparatus as set forth in claim 11, wherein the 20. The apparatus as set forth in claim 11, wherein the 
pre-specified range of values indicative of a blocking point processor adjusts adjacent and neighboring pixels by pro- 
comprises an upper and lower limit, said upper and lower 30 portionate amounts of the adjustment value. 

limits selected such that normal minor image variation 21. The apparatus as set forth in claim 11, wherein the 
between pixels are not detected as blocking points and processor adjusts adjacent and neighboring pixels by pro- 
significant image variations are not detected as blocking portionate amounts of the adjustment value proportionate to 
points. a distance each pixel of the adjacent and neighboring pixel 

15. The apparatus as set forth in claim 11, wherein the 35 is from the blocking point, 
pre-specified range of values indicative of a blocking point 

comprises an upper limit and lower limit, said lower limit ***** 
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